Add mako Docker image for performance testing by saintstack · Pull Request #13159 · apple/foundationdb

saintstack · 2026-05-06T00:45:56Z

Add an opt-in mako Docker image target to the build system. The mako benchmark binary is packaged into a minimal container for use in k8s performance regression tests. Build with BUILD_MAKO=1 to include it.

Add mako stage to Dockerfile
Add mako to build-images.sh (gated on BUILD_MAKO=1)
Strip debug symbols from mako binary

(Mako is FDB's native C benchmark tool. It's a simple load generator that connects directly to FDB using the C client library and runs configurable workloads — random reads, writes, inserts, range scans — against a keyspace. It's built from source in bindings/c/test/mako/. Compared to YCSB (Java-based), mako is lighter weight, lower overhead, no JVM, and already part of the FDB codebase.)

Copilot

Pull request overview

Adds an opt-in Docker image target for the mako benchmark so it can be deployed in Kubernetes-based performance/regression testing flows, enabled via BUILD_MAKO=1.

Changes:

Add a new mako stage/target to packaging/docker/Dockerfile that packages an architecture-specific mako binary.
Gate inclusion/build/push behavior for the mako image in packaging/docker/build-images.sh behind BUILD_MAKO=1.
Strip debug symbols for the mako target in the C bindings CMake build.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
packaging/docker/Dockerfile	Adds a new `mako` image stage that installs `mako` and runs it as `fdb`.
packaging/docker/build-images.sh	Adds `BUILD_MAKO` gating to include `mako` in website prep and image build list.
bindings/c/CMakeLists.txt	Strips debug symbols from the `mako` target during packaging.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

saintstack · 2026-05-06T05:20:42Z

+    chmod +x /usr/bin/mako && \
+    rm -rf /tmp/*
+
+RUN mkdir -p /var/log/fdb-trace-logs


Implemented.

foundationdb-ci · 2026-05-06T00:52:17Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 3e6c48b
Duration 0:06:11
Result: ❌ FAILED
Error: Error while executing command: cmake -S . -B build_output -D USE_CCACHE=ON -D USE_WERROR=OFF -D BUILD_DOCUMENTATION=ON -DOPEN_FOR_IDE=ON -G Ninja. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T01:30:38Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 3e6c48b
Duration 0:44:31
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T02:09:06Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 3e6c48b
Duration 1:22:50
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-06T03:03:34Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 3e6c48b
Duration 2:17:29
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T03:18:22Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 3e6c48b
Duration 2:32:14
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T03:20:37Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 3e6c48b
Duration 2:34:32
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T03:27:50Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 3e6c48b
Duration 2:41:40
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Add an opt-in mako Docker image target to the build system. The mako benchmark binary is packaged into a minimal container for use in k8s performance regression tests. Build with BUILD_MAKO=1 to include it. - Add mako stage to Dockerfile - Add mako to build-images.sh (gated on BUILD_MAKO=1) - Strip debug symbols from mako binary

foundationdb-ci · 2026-05-06T05:45:33Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:23:54
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T05:56:31Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 0:34:53
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T06:07:00Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 248a8a6
Duration 0:45:22
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T06:08:28Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 0:46:48
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T06:18:01Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:56:24
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T06:18:11Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:56:35
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T06:31:05Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:09:28
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

ploxiln · 2026-05-06T14:38:17Z

+    fi; \
+    cp "/tmp/mako.${FDB_ARCH}" /usr/bin/mako && \
+    chmod +x /usr/bin/mako && \
+    rm -rf /tmp/*


It's worth noting that this rm doesn't save any space, because these files were added in the previous layer (directive). But I don't see a better way 🤷

saintstack · 2026-05-06T14:54:24Z

RandomSeed="940731340" SourceVersion="248a8a68da91efb45ac1e66f1387dc7b435f6917" Time="1778047238" BuggifyEnabled="0" DeterminismCheck="0" FaultInjectionEnabled="1" TestFile="tests/fast/MaxGrvQueueDelay.toml"

foundationdb-ci · 2026-05-06T15:18:42Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:24:00
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T15:30:41Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 0:36:03
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T15:36:24Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:41:47
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T15:40:43Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 248a8a6
Duration 0:46:04
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-06T15:41:48Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 0:47:08
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-07T21:51:01Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 248a8a6
Duration 0:45:40
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-07T22:07:25Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 1:02:07
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-07T22:15:58Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:10:38
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-07T22:24:56Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:19:32
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-07T22:27:07Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:21:45
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

saintstack · 2026-05-08T17:29:50Z

Mac builds stepping on each other?

foundationdb-ci · 2026-05-08T17:38:17Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 0:08:17
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

ploxiln · 2026-05-08T17:38:37Z

Mac builds stepping on each other?

Yeah ... I'm not sure how ... the exclusion lock thing seems to work, only one build runs at a time, but sometimes we see strangely missing files, almost like an rm -rf cleanup was run concurrently ...

foundationdb-ci · 2026-05-08T17:55:16Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:25:14
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

ploxiln · 2026-05-08T17:59:05Z

Mac builds stepping on each other?

Yeah it looks like the background-ssh-process lock is terminating in the middle of the build sometimes. The idea there was for the lock to be released if the builder completely died, killing the background ssh process as well. Hmm, could change that for a timestamp check, assume lock is held for up to 1hr (unless released early by healthy builder)

foundationdb-ci · 2026-05-08T18:17:11Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 248a8a6
Duration 0:47:11
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

ploxiln · 2026-05-08T18:27:11Z

I've temporarily removed the requirement of macOS builds to pass

saintstack · 2026-05-08T18:46:49Z

Yeah it looks like the background-ssh-process lock is terminating in the middle of the build sometimes. The idea there was for the lock to be released if the builder completely died, killing the background ssh process as well. Hmm, could change that for a timestamp check, assume lock is held for up to 1hr (unless released early by healthy builder)

@ploxiln poking around, mac builds seem to be having issues... compile fails because missing dirs... likely because another concurrent build removed them (didn't look closely). I did remove a /tmp/ci.lock that was a few days old from the m1.

foundationdb-ci · 2026-05-08T18:51:27Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:21:28
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-08T19:09:48Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:39:48
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T19:16:43Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:46:43
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

saintstack · 2026-05-08T19:17:58Z

Definetly unrelated RandomSeed="2072652414" SourceVersion="248a8a68da91efb45ac1e66f1387dc7b435f6917" Time="1778265802" BuggifyEnabled="1" DeterminismCheck="0" FaultInjectionEnabled="1" TestFile="tests/fast/AtomicBackupCorrectness.toml"

foundationdb-ci · 2026-05-08T19:49:37Z

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Commit ID: 248a8a6
Duration 0:31:23
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T20:02:19Z

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Commit ID: 248a8a6
Duration 0:44:07
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T20:26:51Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 1:08:37
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T20:39:45Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 3:09:44
Result: ❌ FAILED
Error: Error while executing command: # compatible with bash and zsh ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} ' set -e -o noclobber I=0 while ! (echo $$ >/tmp/ci.lock) 2>/dev/null ; do if ((I++ > 10800)); then # 3 hours echo "timeout waiting for ci.lock after $I seconds" exit 1 fi if ! ps -p "$(cat /tmp/ci.lock)" >/dev/null; then # racy but sleep so chances are low echo "lock holder may be gone ..." sleep $((8 + RANDOM % 8)) if ! ps -p "$(cat /tmp/ci.lock)" >/dev/null; then echo "lock holder is gone" rm -f /tmp/ci.lock else echo "new lock holder" fi fi sleep 1 done echo "acquired ci.lock" ' . Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T20:39:47Z

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:21:32
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2026-05-08T20:40:54Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 248a8a6
Duration 1:22:43
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} rm -rf foundationdb. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T21:07:02Z

Result of foundationdb-pr-clang on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:48:49
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2026-05-08T21:09:23Z

Result of foundationdb-pr on Linux RHEL 9

Commit ID: 248a8a6
Duration 1:51:08
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

saintstack requested review from Copilot and ploxiln May 6, 2026 00:46

Copilot started reviewing on behalf of saintstack May 6, 2026 00:46 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

saintstack force-pushed the mako branch from 3e6c48b to 248a8a6 Compare May 6, 2026 05:21

ploxiln reviewed May 6, 2026

View reviewed changes

ploxiln approved these changes May 6, 2026

View reviewed changes

saintstack closed this May 6, 2026

saintstack reopened this May 6, 2026

saintstack closed this May 8, 2026

saintstack reopened this May 8, 2026

saintstack closed this May 8, 2026

saintstack reopened this May 8, 2026

saintstack merged commit 77eff92 into apple:main May 8, 2026
6 of 7 checks passed

Conversation

saintstack commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

saintstack May 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

Uh oh!

ploxiln May 6, 2026

Choose a reason for hiding this comment

Uh oh!

saintstack commented May 6, 2026

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang-ide on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang on Linux RHEL 9

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 6, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

Uh oh!

foundationdb-ci commented May 7, 2026

Result of foundationdb-pr-clang-arm on Linux CentOS 7

Uh oh!

foundationdb-ci commented May 7, 2026

Result of foundationdb-pr-macos on macOS Ventura 13.x

saintstack commented May 6, 2026 •

edited

Loading